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ADAPTIVE COLOR SUPER 
RESOLUTION THRESHOLDING 

Background of the Invention 
Field of Invention 

[0001] This invention pertains to micro-adaptive thresholding of a color image. 

Description of Related Art 

[0002] U.S. Patent No. 4,442,544 to Moreland et ai., incorporated herein by reference in its 
entirety, discloses a micro-adaptive thresholder for processing image pixels. As 
discussed in the '544 patent, a portion or block of one or more image pixel lines is 
examined. The maximum and minimum pixel density of the examined portion or block is 
determined. The difference between the maximum and minimum pixel density in the 
portion or block is then determined. That difference is multiplied by a predetermined 
constant, which represents a desired change in threshold level, to provide a base 
threshold. The base threshold is differenced from the maximum pixel density to provide 
an optimum threshold. 

[0003] U.S. Patent No. 5,778,092 to MacLeod, et al., incorporated herein by reference in its 
entirety, discloses a technique for compressing a color or gray scale pixel map 
representing a document. The '092 patent uses a thresholding technique by generating a 
bitmap of binary values. The thresholding process is adaptive and is based on the pixel 
values in a local neighborhood around a target pixel. The process, shown in Figs. 6A and 
6B of the '092 patent, involves generating a sum of all pixels which differ in density by a 
certain maximum value. An average value of all the pixel intensities (or densities) that 
contributed to the maximum difference is then found. This average value of maximum 
pixel intensity (or density) difference is used as the threshold. 
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Summary of the Invention 

[0004] The thresholding methods and systems disclosed in the '544 and '092 patents do not 
take into consideration the chromaticity characteristics of colored image pixels in any 
significant manner. However, many color images contain portions, such as, for example, 
edges, which are not properly binarized by the techniques disclosed in the '544 and '092 
patents. This occurs because the thresholding systems and methods of the '544 and '092 
patents only use luminance, brightness, density values or some approximation of these 
values, such as, for example, L in the L*,a*,b* color specification system, Y in the X,Y,Z 
color specification system, or Y in the CMYK color specification system, as the gray 
reference. However, the edge portions may often be expressed mostly as a chrominance 
change, rather than a luminance, brightness or density change. 

[0005] This invention provides systems and methods that combine the information from all 
color channels to produce a high quality gray channel to be used for binarization. 

[0006] This invention separately provides systems and methods that generate an adaptive 
threshold based on a pixel values derived from three-component or four-component 
color pixel information. 

[0007] In one exemplary embodiment, this is accomplished by determining a single- 
component gray pixel value based on three-component or four-component color pixel 
information to provide an adaptive threshold for image processing color pixel 
information. 

[0008] These and other features and advantages of the invention are described in, or are 

apparent from, the following detailed description of various exemplary embodiments of 
the systems and methods according to th is invention. 

Brief Description of the Drawings 

[0009] Various exemplary embodiments of this invention will be described in detail, with 
reference to the following figures, wherein: 

[001 0] Fig. 1 depicts a conventional color coordinate specification system; 

[001 1] Fig. 2 depicts a number of image pixel vector quantities used to determine an 
adaptive threshold in the color space illustrated in Fig. 1 ; 
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[001 2] Fig. 3 depicts an image pixel neighborhood including the central pixel in the 
neighborhood; 

[001 3] Fig. 4 is a flowchart depicting one exemplary embodiment of a method for combining 
color information a plurality of color channels and a threshold to form a high quality gray 
image according to this invention; 

[0014] Fig. 5 depicts a first exemplary embodiment of a system that adaptively thresholds a 
pixel image; and 

[001 5] Fig. 6 depicts a second exemplary embodiment of a system that adaptively thresholds 
a pixel image. 

Detailed Description of Exemplary Embodiments 

[001 6] This invention provides systems and methods for thresholding image pixels in 

general, and in particular for adaptively thresholding image pixels in response to changes 
in color content of an image. 

[001 7] This invention describes systems and methods of incorporating information from the 
color components of an image, such as three tristimulus values, or three tristimulus 
values and an achromatic value, such as, for example, black, and using an adaptive 
threshold to produce a single-component high quality gray signed image. This single- 
component high quality gray signed image can then be supersampled and binarized by 
comparing the sampled values to zero. Various systems and methods according to this 
invention determine the distance of a given pixel in a color space defined in terms of the 
color components, or coordinates derived from the color components, from an idealized 
sector plane which segments the color space into two regions. 

[001 8] The sector plane may be created for mixed raster content or simply a binary image 
for facsimile by binarizing an image. Binarizing an image involves comparing each pixel 
to a threshold and producing a 1 if the pixel's density or luminance or brightness equals 
or exceeds the threshold density, luminance or brightness, and producing a 0 otherwise. 
Moreover, because a color source image has multiple bits per pixel, such as, for example, 
8 bits per pixel, useful higher resolution images can be produced by scaling up the gray 
image prior to thresholding. Typically, bilinear interpolation is used to produce the 
higher resolution gray image. The process is made adaptive by making the threshold a 
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function of the neighborhood pixels, such as, for example, making the threshold an 
average of the maximum and minimum density, luminance or brightness of the 
neighboring pixels. 

[001 9] Color images typically have three color components (e.g., X,Y,Z) or four color 

components (e.g., C,M,Y,K) and may have fewer or more color components. Figs. 1 and 2 
show one example of a color coordinate system which defines a color space. Fig. 1 is a 
perspective view of the CIE 1 931 XYZ tristimulus color space. Each tristimulus value (X,Y, 
or Z) corresponds to a primary color which can be used to describe a color located in the 
XYZ color space. The shaded triangle is a plane which is defined by the equation X-f Y-f-Z 
= 1 . Fig. 2 shows a portion of the shaded triangle. The X and Y tristimulus values in the 
shaded triangle plane are x and y chromaticity coordinates, where x = X/(X+Y+Z) and y 
- Y/(X+Y+Z). 

[0020] The systems and methods according to this invention allow the information from the 

color channels to be meaningfully combined to produce a high quality gray channel to be 

used for binarization. The characteristics of the color channels may be specified in any 

known or later-developed color space, such as, for example, the color space defined by 

the CIE 1 931 color specification system. Fig. 2 shows parameters used in this invention in 

a given color space. The dependent maximum, D , is the pixel value in the local 

max 

neighborhood which has the highest luminance. Luminance is also referred to 

conventionally as lightness, brightness, or value. The dependent minimum, D ^ is the 

pixel value in the local neighborhood which has the lowest luminance. D and D 

are three-component vectors. For example, where each vector D and D has an 

max mm 

x, y and z component in the XYZ color space. Any particular color specification system 
may be used. Certain color specification systems are particularly convenient, in that, in 
these certain color specification systems, the maximum luminance can be found by 
examining only one component, i.e., the luminance (e.g., the value, lightness or 
brightness). 

[0021] 

The systems and methods of this invention establish a pixel neighborhood or image 
portion or block. In various exemplary embodiments of the systems and methods of this 
invention, this image pixel neighborhood, image portion or block is a 5x5 pixel window. 
This exemplary embodiment is shown in Fig. 3. It should be understood that a one- 
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dimensional pixel neighborhood could be selected, or a smaller or larger two- 
dimensional neighborhood could be selected, or a three-dimensional pixel neighborhood 
could be selected, depending on the characteristics of the image being sampled. 

[0022] Fig. 4 is a flowchart outlining one exemplary embodiment of a method for combining 

color information a plurality of color channels and a threshold to form a high quality gray 

image according to this invention. As shown in Fig, 4, beginning in step SI 00, operation 

continues to step SI 1 0, where a first or next pixel in the image is selected as the current 

target pixel, and a window of image pixels surrounding the current target pixel is 

determined. Then, in step SI 20, the color attributes, such as, for example, the 

tristimulus values R, G and B, of each color image pixel in the current window are 

measured. Next, in step SI 30, color space coordinates, including luminance, brightness 

and/or optical density, of all pixels in the current window, including a center or target 

pixel C , are determined from the measured color attributes, 
pix 

[0023] Typically, such color spaces include those defined using the CIE L*,a*,b* coordinate 

system, the CIE uniform chromaticity scale (UCS) system, or another known color space, 

such as, for example, the CIE X,Y,Z system; the C,M,Y system; the C,M,Y,K system, the 

rCb'Cr' system, Y*,u>*; U*,V*,W*; S, 9 , W*; C,M,Y; C,M,Y,K; Y'Cb'Cr; NTSC Y,I,Q; H,S,I; 

H,S,V; CIE R,G,B; and NTSC R Ki ,C ki ,B Ki systems. Typically, in these systems, one of the 

N N N 

coordinates corresponds to luminance, brightness and/or density. In the CIE L* f a*,b* 
system, L* represents luminance. In the CIE X,Y,Z system, Y represents luminance. Thus, 
the systems and methods of this invention obtain the luminance of each pixel in 
neighborhood defined by the window in the context of the given color space, i.e., in 
terms of not only luminance but also chromaticity. Operation then continues to step 
SI 40. 

[0024] 

in step SI 40, the pixel value of the most luminant pixel in the current window is 

denoted D and the pixel value of the least luminant pixel is denoted D-D 

fp^x min max 

and D are vectors with three or four color components. Moreover, for images with 
min 

higher levels of noise, it may be desirable to preprocess the image pixels used to 

determine D and D by, for instance, blurring or smoothing. For simplicity, a 
max min 

three-component space will be described, although any number of vector components 

may be used. As described above, D and D . are shown as vectors in the 
1 max mm 
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represented color space. Operation then continues to step SI 50. 

[0025] In various exemplary embodiment that use a three-color component, each vector has 
an x component, a y component and a z component. Although the particular color space 
used is not important, some color spaces, such as, for example, CIE L*,a*,b*; Y'Cb'Cr'; 
and others discussed above, are convenient to use because the maximum and minimum 
pixel luminance can be found by only examining one of the three color components. 

[0026] In step SI 50, the vector average, D , , of D and D .is determined as: 
1 J thr max mm 

[0027] D = (D +D . )/2. 

thr max mm 

[0028] Next, in step SI 60, the vector difference, D , between D and D is 
L J r nrm max mm 

determined as: 

[0029] D = (D -D . ). 

nrm max mm 

[0030] jn steps S1 7Q and S1 8Q c j etermjnat j ons are mac |e whether to use adaptive 

thresholding or static thresholding. The decision to use adaptive thresholding is made 
based on two tests. In step SI 70, the first test determines whether the luminance 

component of the difference vector D is greater than some luminance noise threshold 

nm 

T T is an empirically determined parameter which may be determined in advance or 
n n 

dynamically during the thresholding process. If the luminance component of the 

difference vector D is greater than a particular luminance noise threshold T , 
nm n 

control proceeds to step SI 80. If not, control proceeds to step SI 90. In step S 1 80, the 

second test determines whether the overall vector magnitude of D ^ exceeds some 

"estimated magnitude" threshold T . To simplify the second test, the estimated 

em 

magnitude threshold may be approximated by the well known Euclidian difference or the 

Manhattan difference. The Manhattan distance estimate is the sum of the absolute values 

of the vector components. If the overall vector magnitude of D exceeds some 

nm 

"estimated magnitude" threshold T , control proceeds to step S200. If not, control 

em 

proceeds to step SI 90. If both tests are passed, i.e., if the determinations of both steps 
SI 70 and 1 80 are "yes," then control proceeds to step S200 where a dynamic 

determination of G is made based on adaptive thresholding is made. However, if 
pix 

either test fails, operation continues to step SI 90, where a G . determination is made 

pi x 

based on static thresholding. In step SI 90, G is typically determined to be either a 

r IX 
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plus 1 or a minus 1 . 

[0031] In step SI 90, a static luminance threshold is subtracted from the luminance 

component of C to produce G , In step S200, dynamic thresholding is active and 
PIX PIX 

the signed single component gray pixel value, G , is determined as: 

[0032] G . =((C . -D )*D )/S 
pix pix thr nrm 

[0033] where: 

[0034] S is a predetermined scale value, such as, for example, 256 when 8-bit samples are 

used, which reduces the overall magnitude of G . for convenient storage; and 

pix 

[0035] " * " is the vector dot product operator. 

[0036] It is useful to remember that a dot product, also called an inner product, of two 

vectors is (1) the product of their magnitudes times the cosine of the angle between them 
or (2) the magnitude of a first vector times the projection of the other vector along the 
first vector. 

[0037] This exemplary method determines the signed distance of G . from the color space 

pix 

plane which contains D and to which D is perpendicular. G . is represented in 
K thr nrm pix 

Fig. 2 as the magnitude of the solid vector extending to the point C from the 

threshold plane that contains D , . That magnitude, and therefore G , , is the dot 

thr pix 

product of (C - D ) and D . Fig. 2 shows these quantities, as well as the angle 
pix thr nrm 

9 between D and (C . » D ). 

nrm pix thr 

[0038] G is negative below the threshold plane and is positive above the threshold plane, 
pix 

Thus, zero can be used as a binarizing value of G . . Where the vector difference 

pix 

between D and D falls below some system noise threshold, T , or less than an 
max min n 

estimate of an edge magnitude of difference T , the systems and methods of this 
^ em 

invention fall back to using a static, single component thresholding. In this manner, the 
systems and methods according to this invention can avoid binarizing image noise. 

[0039] Next, in step S21 0, a determination is made whether any other pixels in the image 
need to be analyzed. If so, operation returns to step S1 10. Otherwise, operation 
continues to step S220, where the method ends. 
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[0040] Fig. 5 shows a first exemplary embodiment of a micro-adaptive color image 

thresholding system 300 according to this invention. As shown in Fig. 5, the a micro- 
adaptive color image thresholding system 300 includes an interface 310, a controller 
circuit or software routine or element 320, a memory 330, an image pixel selector circuit 
or software routine or element 340, a color attribute determining device or circuit or 
software routine or element 350, a color coordinate selector circuit or software routine or 
element 360, a vector analyzer circuit or software routine or element 370, a vector 
average circuit or software routine or element 380, a static G determination circuit or 
software routine of element 385, a vector difference determining circuit or software 
routine or element 390, and a dynamic G determination circuit or software routine or 
element 395. The system elements 300-395 are interconnected to each other by a bus or 
application programming interface (API) 200. The interface 31 Ointerconnects an image 
date source 100 and an image data sink 110 with the system elements 320-395. In 
general, the image data source 1 00 can be any one of a number of different sources, 
such as a scanner, a digital copier, a facsimile device that is suitable for generating 
electronic image data, or a device suitable for storing and/or transmitting electronic 
image data, such as a client or server of a network, or the Internet, and especially the 
World Wide Web. The color measurement device 350, which may be integrated with the 
image data source 1 00, such as, for example, a facsimile machine or image scanner, 
measures the color attributes, such as tristimulus values of color image pixels. 

[0041] The controller 320 controls the operation of the system elements 310 and 330-395. 

The color coordinate selector circuit or software routine or element 360 converts these 

tristimulus values to color coordinates, e.g., L*,a*, b* for each pixel, including a 

neighborhood center pixel. The vector analyzer circuit or software routine or element 

370 determines vectors D and D . for the neighborhood pixels. The vector 

max mm 

determining circuit or software routine or element 350 determines the average of D 

and D . The vector difference determining circuit or software routine or element 390 
min 

determines the difference between D and D . . The signal gray value determining 

max mm 

circuit or software routine or element 395 determines a signed gray channel value and 
sets or changes the threshold accordingly. 

[0042] ^ 6 shows a second exemplary embodiment of a system according to this 

invention, as shown in Fig. 6, replaces each of the separate system elements 310-395, as 
APP ID=09682912 Pa 8 e 8 



shown in Fig. 5, with a central processing unit 400, having ROM memory 401 and RAM 
memory 402. As shown in Fig. 6, the image data source 1 00 can be implemented, for 
example, using one or both of a facsimile machine 102 and a scanner 104. 

[0043] The micro-adaptive color image thresholding systems 300 and/or 400 according to 
this invention are preferably implemented on a programmed general purpose computer. 
However, the micro-adaptive color image thresholding systems 300 and/or 400 
according to the invention can also be implemented on a special purpose computer, a 
programmed microprocessor or microcontroller and peripheral integrated circuit 
elements, an ASIC or other integrated circuit, a digital signal processor, a hardwired 
electronic or logic circuit such as a discrete element circuit, a programmable logic device 
such as a PLD, PLA, FPGA or PAL, or the like. In general, any device, capable of 
implementing a finite state machine that is in turn capable of implementing the 
flowcharts shown in Fig. 4, can be used to implement a system according to this 
invention. 

[0044] The memory elements 330, 401 and 402 can be implemented using any appropriate 
combination of alterable, volatile or non-volatile memory or non-alterable, or fixed, 
memory. The alterable memory, whether volatile or non-volatile, can be implemented 
using any one or more of static or dynamic RAM, a floppy disk and disk drive, a writable 
or re-rewriteable optical disk and disk drive, a hard drive, flash memory or the like. 
Similarly, the non-alterable or fixed memory can be implemented using any one or more 
of ROM, PROM, EPROM, EEPROM, an optical ROM disk, such as a CD-ROM or DVD-ROM 
disk, and disk drive or the like. 

[0045] It should be understood that each of the micro-adaptive color image thresholding 
systems 300 and 400 shown in Figs. 5 and 6, respectively, can be implemented as 
portions of a suitably programmed general purpose computer. Alternatively, each of the 
systems 300 and /or 400 shown in Figs. 5 and 6, respectively, can be implemented as 
physically distinct hardware circuits within an ASIC, or using a FPGA, a PDL, a PLA or a 
PAL, or using discrete logic elements or discrete circuit elements. The particular form 
each of the circuits will take is a design choice and will be obvious and predictable to 
those skilled in the art. 



[0046] 
APP ID 



Moreover, the micro-adaptive color image thresholding systems 300 and/or 400 can 
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be implemented as software executing on a programmed general purpose computer, a 
special purpose computer, a microprocessor or the like. In this case, the micro-adaptive 
color image thresholding systems 300 and/or 400 can be implemented as a routine 
embedded in a printer driver, as a resource residing on a server, or the like. The micro- 
adaptive color image thresholding systems 300 and/or 400 can also be implemented by 
physically incorporating it into a software and/or hardware system, such as the hardware 
and software systems of a printer or a digital photocopier. 

[0047] The methods and systems of this invention represent an improvement over the prior 
art thresholding methods and systems mentioned in the Background of the Invention 
section, above. Various exemplary embodiments of the methods and systems of this 
invention produce a smooth, continuous function around zero which incorporates 
contributions from all color components of the pixel image and continuously adjusts the 
signed weights of those contributions based on the correlation of each to pixel 
luminance. 

[0048] Prior thresholding systems and methods, including those discussed above, were 

based solely on achromatic or gray scale luminance, whereas the thresholding systems 
and methods of this invention take into consideration one or more chromaticity 
characteristics, such as for example, the hue or color family, and/or saturation or 
chroma, of the image pixels. This is especially important in image areas where image 
contrast is expressed mostly as a difference in the saturation, i.e., chroma, or the color 
family, i.e., hue between adjacent pixels. 

[0049] Systems and methods of thresholding which include only using luminance or some 
approximation to luminance, such as Y or G, as the gray reference can miss contrast 
differences or changes expressed mostly as saturation or hue differences or changes. 

[0050] While the invention has been described in conjunction with the exemplary 

embodiments outlined above, it is evident that many alternatives, modifications and 
variations will be apparent to those skilled in the art. Accordingly, the exemplary 
embodiments of the invention, as set forth above, are intended to be illustrative, not 
limiting. Various changes may be made without departing from the spirit and scope of 
the invention. 



APP ID=09682912 



Page 10 of 21 



